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Abstract 

^ The systematic biases and errors seen in people's probability judg- 

r—i ments are typically taken as evidence that people do not reason about 

probability using the rules of probability theory. We show the contrary: 
that these biases are a consequence of people correctly following proba- 
bility theory, but with random variation or noise affecting the reasoning 
j—{ process. Taking Pe(A) to represent a person's estimate for the proba- 

T—i bility of some event A, this random variation account predicts that on 

CN average P E (A) + P E (B) - P E (AV B) - P E (A A B) = for all pairs of 

events A, B, just as required by probability theory. Analysing data from 
an experiment asking people to estimate such probabilities for a number 
• rH of pairs A, B we find striking confirmation of this prediction. 

X 

H 

1 Introduction 



The ability to reason with uncertain knowledge (that is, to reason with prob- 
abilities) is crucial to the ability to survive and prosper in "an ecology that is 
of essence only partly accessible to foresight" ( |Brunswik 1955). It is therefore 



reasonable to expect that humans, having prospered in such an ecology, would 
be able to reason about probabilities extremely well: any ancestors who could 
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not reason effectively about probabilities would not survive long, and so the bio- 
logical basis of their reasoning would be driven from the gene pool. Probability 
theory provides a calculus of chance describing how to make optimal predic- 
tions under uncertainty: taking the argument one step further, it is reasonable 
to expect that our probabilistic reasoning will follow the rules of probability 
theory. 

The conventional view in current psychology is that this expectation is 
wrong. Instead, the dominant position is that 

In making predictions and judgments under uncertainty, people do 
not appear to follow the calculus of chance or the statistical theory 
of prediction. Instead they rely on a limited number of heuristics 
which sometimes yield reasonable judgments and sometimes lead to 



severe and systematic errors ( Tversky and Kahneman 1973 p. 237) 



This conclusion is based on series of systematic and reliable biases in people's 
judgements of probability, many identified in the 1970's and 1980's by Tversky, 
Kahneman and colleagues. The heuristics and biases approach has reached a 
level of popularity rarely seen in psychology, with Kahneman recieving a Nobel 
Prize for his work in this area, and with numerous popular psychology books and 
newspaper articles presenting this view to the general public. Indeed, the idea 
that people cannot reason with probabilities has become a widespread truism: 
for example, the Science Gallery in Dublin recently announced an exhibition on 
risk which it describes as "enabling visitors to explore our inability to determine 
the probability of everything from a car crash to a coin toss" (The Irish Times 
- Thursday, October 11, 2012). 

In this paper we describe a very simple alternative to the heuristics and biases 
view. In this account people reason about probability according to the rules of 
probability theory, but are subject to random variation or noise in this reasoning 
process. While it may seem that noise in the workings of a rational system 
will result in "nothing more than error variance centered around a normative 



response" ( Shafir and Leboeuf 2002), in fact there are a number of ways in 
which random variation can produce systematic and reliable deviations from the 
normatively correct responses. These systematic deviations often correspond to 
the observed patterns of bias seen in people's probability judgments. 

This 'probability theory plus noise' account of biases in human probabilistic 
reasoning has a critical characteristic that differentiates it from the heuristic- 
saccount: the random variation account predicts that there should, on average, 
be no systematic bias or error in people's judgements for some specific proba- 
bilities. Taking P(A) to represent the objective, true probability of some event 
A, Pe{A) to represent a reasoner's estimate of that probability, and Pe(A) to 
represent the mean or expected value of Pe{A) (the average estimate of the 
probability of event A), this random variation account predicts that 

P E ~(A)+P E {B)-P E {AAB)-P E (A\/B) = P(A)+P(B)-P(AAB)-P(AVB) = 

for all A and B. This prediction follows directly from the familiar 'addition law' 
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of probability theory, which states that 

P(A VB) = P{A) + P{B) - P(A A B) (1) 

(because the size of the set of events Ay B is equal to the size of the set of events 
A plus the size of the set B, minus the union of those two sets). Analysing data 
from an experiment asking people to estimate such probabilities for a number 
of pairs A, B we find strikingly strong confirmation of this prediction, showing 
that people's probability judgments embody the addition law. 

The organisation of the paper is as follows. In the first section we present 
our very simple account and show how it can explain well-established and sys- 
tematic errors in human probabilistic reasoning: conservatism, subadditivity, 
the conjunction error, and the disjunction error. In the second section we de- 
scribe our experimental results confirming the above prediction, and in the final 
section discuss some implications of this result. 



2 The systematic influence of noise 



In this section we present our simple account of the influence of noise on people's 
probability estimates by showing how this account can explain 4 well-known 
patterns of bias in people's probability judgment: conservatism or underconfi- 
dence, subadditivity, the conjunction error, and the disjunction error. Other 
more complex accounts also based on noise have previously been given for each 
of these biases (see Costello. 200 9a|b Erev et al. 1994 Hilbert 2012 1: as far as 
we know this is the first account explaining all 4 biases, and the first to make 
specific predictions about when bias will vanish. 

In discussing the influence of random variation on probabilistic reasoning we 
assume a rational reasoner with a long-term episodic memory. For simplicity 
we assume that, apart from random variation, the reasoner is 'perfect': that is, 
if the reasoner were not subject to random variation then each estimate Pe(A) 
would be equal to the correct probability P(A). We assume a simple form of 
long-term memory containing n episodes where each recorded episode i contains 
a flag fi (A) that is set to 1 if i contains event A and set to otherwise, and the 
reasoner estimates the probability of event A by counting these flags. We assume 
a minimal form of transient error, in which there is some small probability d 
that when the state of some flag fi{A) is read, the value obtained is not the 
correct value for that flag. We take C{A) to be number of flags that were read 
as 1 in some particular query of memory, and Ta be the number of flags whose 
correct value is actually 1. Note that due to random error C(A) — Ta does not 
necessarily hold. 



2.1 Conservatism 

Probabilities range in value between and 1. A large body of literature demon- 
strates that people tend to keep away from these extremes in their probability 
judgments, and so are 'conservative' in their probability assessments. These 
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results show that the closer P(A) is to 0, the more Pe(A) is greater than P{A), 
while the closer P(A) is to 1, the more Pe(A) is less than P(A). Differences 
between true and estimated probabilities are low when P(A) is close to 0.5 and 
increase as P(A) approaches the boundaries of or 1. This pattern was orig- 
inally seen in research on people's revision of their probablity estimates in the 



light of further data (Edwards 1968 ), and was later found directly in probability 



estimation tasks. This pattern is sometimes referred to as underconfidence in 



people's probability estimates (see Erev et al. 1994 for a review) 



Conservatism will occur as a straightforward consequence of random varia- 
tion in our otherwise perfect reasoner. Our reasoner computes Pe{A) by query- 
ing episodic memory to count all episodes containing A and dividing by the 
total number of episodes, giving 

P E (A) = 

n 

Random variation afffects Pe(A) when it causes some flag fi(A) be read incor- 
rectly. The expected value of Pe(A) is given 

— = T A (l -d) + (n- T A )d 
n 

(since on average 1 — d of the T4 flags whose value is 1 will be read as 1, and d 
of the n — Ta flags whose value is will be read as 1 ) . Since we assume that 
if the reasoner were not subject to random variation then each estimate Pe(A) 
would be equal to P{A), we have 

m = £ 

and so 

P~e~{A) = P(A) +d- 2dP(A) (2) 

and the average value of Pe(A) deviates from P(A) in a way that systematically 
depends on P(A). This deviation produces the pattern of conservatism seen in 
people probability judgments. If P(A) — 0.5 this deviation will be 0, if P(A) < 
0.5 then since d cannot be negative we have Pe(A) > P(A) with the difference 
increasing as P(A) approaches 0, and if P(A) > 0.5 then Pe(A) < P(A) with the 
difference increasing as P(A) approaches 1, just as seen in people's probability 
judgments. 

2.2 Subadditivity 

Let Ai . . . A n be a set of n mutually exclusive events, and let A = A\ V . . . V A n 
be the disjunction (the 'or') of those n events. Then probability theory requires 
that 

P{A l ) + ... + P{A n )=P{A) 

Experimental results show that people reliably violate this requirement, and 
in a characteristic way. On average the sum of people's probability estimates 
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for events A\ . . . A n is reliably greater than their estimate for the probability 
of A, with the difference (the degree of 'subadditivity') increases reliably as n 
increases. An additional, more specific pattern is also seen: for pairs of mutually 
exclusive events Ai and A 2 whose probabilities sum to 1 we find that the sums 
of people's estimates for A\ and A 2 are normally distributed around 1, and so 
on average this sum is equal to 1 just as required by probability theory. This 



pattern is sometimes referred to as 'binary complementarity' (see Tversky and 



Koehler 1994 for a detailed review of these results). 



Again, these patterns of subadditivity occur as a straightforward conse- 
quence of random variation in our otherwise perfect reasoner. From Equation 
[2] we have 



P E (A t ) 



P E (A n ) =P(A 1 ) 



d-2dP(A 1 )- 

d - 2dP{A 2 y 



P(A n ) +d- 2dP(A n ) 



or rearranging 

P^(A 1 )+. . .+PE~(A n ) = {P[Ax 
Since by definition 



we can rewrite this as 
PsiAt) 



.. + P{A n ))+nd-2d{P{A l ) 
P{A 1 ) + ... + P(A n ) = P{A) 

-... + PE{A n ) = P(A) +nd- 2dP(A) 



P{A n )) 



Taking the difference between this expression and that for Pe{A) (from Equa- 
tior|2]) we get 

Pe(Ai) + . . . + PE(A n ) - P^{A) = P(A) + nd- 2dP{A) - P(A) -d+ 2dP(A) 

= (n-l)d 

and so this difference increases as n increases, producing a pattern of subaddi- 
tivity just as seen in people's probability judgments. In the case of two mutually 
excusive events Ai and A 2 whose probabilities sum to 1, from Equation [2] we 
get 

P^{Ax) + PE(A 2 ) = P(A 1 ) +d- 2dP(A 1 ) + P{A 2 ) +dl- 2dP(A 2 ) 
= P(A 1 ) + P(A 2 ) + d{2 - 2(P(A 1 ) + P{A 2 )) 
= P(A 1 ) + P(A 2 ) = 1 

producing just the pattern of binary complementarity seen in people's proba- 
bility judgments. 
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2.3 Conjunction and disjunction errors 

The above two biases concern averages of estimated probability values. The 
next two biases concern differences between probability estimates. Let Ai and 
A 2 be any two events ordered so that P(A\) < P(A 2 ). Then probability theory 
requires that 

P(Ai A A 2 ) < P(Ai) 

This following from the fact that A\ A A2 can only occur if A\ itself occurs. 
People reliably violate this requirement for some events, giving probability esti- 
mates for conjunctions that are greater than the estimates they gave for one or 
other constituent of that conjunction. This 'conjunction error' does not occur 
for all or even most conjunctions (people correctly follow the rules of probability 
theory for most conjunctions). Numerous experimental studies have shown that 
the occurence of this error depends on the probabilities of Ai and A 2 . In par- 
ticular, the greater the difference between P(Ai) and P(A 2 ), the more frequent 
the conjunction error is, and the greater the conditional probability P(A\\A 2 ), 
the more frequent the conjunction error is. The frequency of the error can be 



high when these two conditions hold (see Costello 2009a for a detailed review). 

A similar pattern occurs for people's probability estimates for disjunctions. 
Since Ai V A 2 necessarily occurs if A 2 itself occurs, probability theory requires 
that 

P{At V A 2 ) > P(A 2 ) 

must always hold. People reliably violate this requirement for some events, giv- 
ing probability estimates for disjunctions that are less than the estimates they 
gave for one or other constituent. Just as for the conjunction error, the greater 
the difference between P{A\) and P(A 2 ), and the higher the estimated condi- 
tional probability P(Ai\A 2 ), the higher the rate of occurence of the disjunction 
error. Studies which examine the rate of both errors show a strong correlation 
between the frequency of the conjunction error for a given pair of events and 



the frequency of the disjunction error for that same pair (see Costello 2009b 
for a review). 

Again, these patterns occur as a straightforward consequence of random 
variation in our otherwise perfect reasoner. Since the reasoner is subject to 
random variation we can write 

P E {A l ) = P{A l ) + d-2dP{A l ) 

and 

P^(A 1 A A 2 ) = P{A X AA 2 ) + d- 2dP(A 1 A A 2 ) 

and we see that the reasoner's estimates for P{A\) and P{A\ A A 2 ) will vary 
around those means. More specifically the reasoner's estimates for those prob- 
abilites at any given moment will be equal to 

Pe{A x ) =P E {A 1 ) + e Al 

and 

P E (A 1 A A 2 ) = P~^{A 1 A A 2 ) + e AlAA2 
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where e Al and e AlAA2 represent positive or negative random deviation from the 
mean at that time. The conjunction error will occur when 



P E {M) + e Al < P E (A 1 A A 2 ) + e AlAA2 

or, substituting and rearranging, when 

(P(^r) - P(A± A A 2 )){\ - 2d) < e Al AA 2 - e Al (3) 

holds. Given that e Al AA 2 and e Al vary randomly and can be either positive or 
negative, this inequality can hold in some cases. The inequality is most likely 
to hold when P{A{) — P{A\ A A 2 ) is low (because in that situation the left side 
of the inequality is low). Since 

P(A± A A 2 ) = P(Ai\A 2 )P(A 2 ) 

we see that P{A{) - P{A X A A 2 ) is low when P{A{] is low and P(A 1 \A 2 ) and 
P(A 2 ) are high. We thus expect the conjunction error to be most frequent when 
P(Ai) is low and P(Ai\A 2 ) and P(A 2 ) are both high. This is just the pattern 
seen when the conjunction error occurs in people's probability estimates. 

Reasoning in just the same way for disjunctions, we see that the disjunction 
error will occur when 

Pe{A 2 ) + e A . 2 > P~e(Ai V A 2 ) + e AlVA2 

or, substituting and rearranging as before, when 

(P{A 1 V A 2 ) - P(A 2 ))(1 - 2d) < e A2 - e AlVA2 

holds. But from the addition law (Equation [I]) we have 

P(A 1 VA 2 )-P(A 2 ) = P(A 1 )+P(A 2 )-P(A 1 AA 2 )-P(A 2 ) = P(A 1 )-P{A 1 AA 2 ) 

and substituting we see that the disjunction error will occur when 

(P(A X ) - P{A X A A 2 )){\ - 2d) < e A2 ~ e AlVA2 (4) 

and so, just as with the conjunction error, we expect the disjunction error to 
be most frequent when P(Ai) is low and P(Ai\A 2 ) and P{A 2 ) are both high. 
More specifically, since the criteria for the conjunction error and the disjunction 
error are the same (compare the left hand sides of Equations [3] and [4]) we expect 
that on average the rate of occurrence of the disjunction error for a given set 
of items should match the rate of occurrence of the conjunction error of those 
items. Again, this is just the pattern seen when the disjunction error occurs in 
people's probability estimates. 
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3 Predicting unbiased probability estimates 



We saw in the previous section that a simple account where probability estimates 
are normatively correct but influenced by random noise can explain various pat- 
terns of bias in people's probability judgements, and also explain some specific 
situations in which those biases vanish (when probabilities are close to 0.5, for 
conservatism; and when two complementary probabilities sum to 1, for subad- 
ditivity). We now present a third situation in which this account predicts that 
bias will disappear. 

Consider an experiment where we ask people to estimate, for any pair of 
events A and B, the probabilities of A, B, AAB and AwB. For each participant's 
estimates for each pair of events A and B we can compute a derived sum 

X E (A, B) = P E (A) + P E {B) - P E (A A B) — P E (A V B) 

We can make a specific prediction about the average value of X E (A, B) for all 
events A and B. This value will be 



X E (A, B) = P E {A) + P E {B) - P E {A A B) — P E (A V B) 

From Equation [2] we get 

X~e{A, B) =P(A) +d- 2dP{A) 

+ P(B) + d - 2dP(B) 

- P(A AB)-d + 2dP{A A B) 

- P(A V B)-d + 2P(A V B) 

or rearranging 

Xe(A, B) = (1 - 2d){P(A) + P{B) - P{A A B) — P{A V B)) 

However, from the addition law (Equation [I] we get 

P{A) + P{B) - P(A A B) — P(A V B) = 

for all events A and B, and we see that X E (A, B) — 0. Our prediction, therefore, 
is that on average X E (A, B) will be equal to for all pairs of events A and B. 
More specifically, we predict that across a range of judgments for pairs A, B, 
values of X E {A, B) will be normally distributed around a mean of 0. 

We tested this prediction using data from an experiment on conjunction and 



disjunction errors carried out by the first author (Experiment 3 in Costello 



2009b). The original aim of this experiment was to examine an attempt by 
Gigerenzer to 'explain away' the conjunction error as a consquence of people be- 
ing asked to judge the probability of one-off, unique events. Gigerenzer argued 
that from a frequentist standpoint the rules of probability theory apply only to 
repeatable events and not to unique events, and so people's deviation from the 
rules of probability theory for unique events are not, in fact, fallacious. To assess 
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this argument the experiment examined the occurrence of fallacies in probability 
judgements for conjunctions and disjunctions of canonical repeatedly-occurring 
events: weather events such as 'rain', 'wind' and so on. Contrary to Gigeren- 
zcr's argument, participants in these experiments often committed conjunction 
and disjunction errors; these errors thus cannot be dismissed as an artifact of 
researchers using unique events in their studies of conjunctive probability. 

This experiment gathered probability estimates from 83 participants for A,B, 
AAB and Ay B expressions for 12 distinct pairs of events. We briefly present 
the materials and procedure for this experiment below. 

3.1 Materials 

Two sets of weather events we used to construct the materials for this experi- 
ment: the set 'cloudy', 'windy', 'thundery', 'sunny' and the set 'cold', 'frosty', 'sleety' 
These sets were selected to contain weather events with a range of different prob- 
ability values from high (on average participants estimated that 76 days out of 
100 arc cloudy) to intermediate (participants estimated that 38 days out of 100 
are sunny) to low (participants estimated that 12 days out of 100 are thundery). 

From these two sets, conjunctive and disjunctive weather events were formed 
by pairing each member of the first set with every member of the second set and 
placing 'and'/'or' between the elements of each pair as required. Participants 
were asked to judge the probability of all single events in the two sets and all 
conjunctive and disjunctive events constructed from these pairings; there were 
thus 7 single events, 12 conjunctive events and 12 disjunctive events in total. 
Each participant gave probability estimates for all these events. 

3.2 Procedure 

Participants in this experiment were divided into two groups. Participants in 
the first group were asked questions with the form 

• 'What is the probability that the weather will be W on a randomly selected 
day in Ireland?' 

(where W stands for some weather event such as 'sunny', 'windy and sunny', 
'windy or sunny', and so on). Participants in the second group was asked 
questions of the form 

• 'Imagine a set of 100 different days, selected at random. On how many of 
those 100 days do you think the weather in Ireland would be W?' 

(where W again stands for some weather event). Participant responses to both 
frequency and probability rating questions were given on a sliding scale of in- 
tegers ranging from to 100. Participants answering frequency questions were 
told that an integer selected on this scale for a given question represented the 
number of days that participant thought the weather event in question would 
occur. Participants answering probability questions were told that an integer se- 
lected on this scale represented the percentage probability (chance out of 100%) 
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that the weather event in question would occur. Each participant saw events in 
a different random order. 



3.3 Conjunction and disjunction error rates 

A conjunction error was recorded when a participant gave a conjunction a rating 
five points more probable or more frequent (on the to 100 scale) than one or 
other constituent. A disjunction error was recorded when a participant gave 
a disjunction a rating five points less probable or frequent than one or other 
constituent. While errors occured less often in the frequency group than in 
the probability group (confirming previous results showing that the frequency 
format reduces but does not eliminate these errors) , there was a significant error 



rate in both groups (see |Costello 2009b ) 



Since the distinction between frequency and probability groups is not rele- 
vant to our current concerns, we combine the two groups in our analysis. There 
were 996 distinct conjunction responses in the experiment (83 participants xl2 
conjunctions): a conjunction error was recorded in 391 (39%) of those responses. 
There were also 996 distinct disjunction responses: a disjunction error was 
recorded in 388 (39%) of these. There were an average of 4.7 conjunction er- 
rors and 4.7 disjunction errors per participant; of the 83 participants only 1 did 
not produce any error responses. There was a significant correlation between 
the average difference between constituent probability values and the occurence 
of the conjunction fallacy (r=0.92,pj0.01) and and between that difference and 
the occurence of the disjunction fallacy (r = 0.90, p < 0.01), just as predicted. 
There was also a significant correlation between conjunction error rate and dis- 
junction error rate across materials (r ~ 0.83, p < 0.01) showing the predicted 
relationship between the two patterns of error. 



3.4 Values of Xe(A, B) 

For every pair of weather events A, B used in the experiment, each participant 
gave probability estimates for the two constituents A and B, for the conjunction 
Af\B and for the disjunction A\/ B. Each participant was gave these estimates 
for 12 such pairs. For each participant we can thus calculate the value Xe(A, B) 
for 12 pairs A, B, and so across all 83 participants we have 996 distinct values 
of Xe(A, B). Our prediction is that these values will be normally distributed 
around 0. 

The mean value of X(A, B), averaging across all 996 distinct values obtained 
in the experiment, was 0. 66(<S'-D = 27.1). This mean was within 1 unit of 
the predicted mean on the 100 point scale used in the experiment. Figure 1 
shows the raw distribution of these values in the experimental data; Figure 2 
gives a simpler picture, showing the distribution of these values in bins of size 
v — 5 . . . v + 5 for v from —100 to 100. It is clear from these graphs that Xe(A, B) 
is normally distributed around this mean (a similar pattern is seen for a range of 
different bin sizes). The fact that the observed mean differs from the predicted 
mean by less than seven parts in a thousand is a striking confirmation of the 
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Figure 1: Scatterplot showing the frequency of different values of X E (A, B) as 
observed in the experimental data. 



prediction. Using T-tests comparing values of Xe(A, B) with candidate means 
we obtain a 99% confidence interval of (—1.6... 2. 8) for the mean. We can 
thus say with 99% confidence that the population mean for X E (A, B) is within 
2.8 units of the predicted value on the 100 point scale, further confirming the 
prediction. 



4 Discussion 

The above result is based on a specific expression X E {A 1 B) that cancels out 
the effect of noise in people's probability judgements. When noise is cancelled 
in this way we get a mean value for X E {A, B) that is almost exactly equal 
to that predicted by probability theory. This close agreement with probability 
theory occurs alongside reliable and systematic biases towards conjunction and 
disjunction errors in the same probability judgments. These results provide 
strong evidence that, when making judgments under uncertainty, people do in 
fact closely follow the calculus of chance provided by probability theory, and 
that the observed patterns of bias are due to the systematic influence of noise 
on those judgements. 

Note that we are not suggesting that people are consciously aware of the 
equations of probability theory when estimating probabilities. That is evidently 
not the case, given the high rates of conjunction and disjunction errors in peo- 
ple's probability judgments. Indeed we doubt whether any of the participants 
in our experiment were aware of the invariant required by probability theory's 
addition law, or would be able to consciously apply that law to their proba- 
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Figure 2: Scatterplot showing the frequency of different values of Xe(A, B) in 
bins of size v — 5 . . . v + 5 for v from —100 to 100 in steps of 10. 



bility estimations. Instead we propose that people's probability judgments are 
derived from a 'black box' module of cognition that estimates the probability 
of an event A by retreiving (some analogue of) a count of instances of A from 
memory. Such a mechanism is necessarily subject to the requirements of set 
theory and therefore implicitly embodies the equations of probability theory 
(which derive from set theory). 

We expect this probability estimation module to be unconscious, automatic, 
rapid, parallel, relatively undemanding of cognitive capacity, and evolutionar- 
ily 'old' and so shared between humans and animals. In the terms used by 
Stanovich and West Stanovich and West (20001, this is a System 1 process, 



albeit one that is not based on heuristics but instead on the normatively cor- 
rect rules of probability theory. Support from this modular view comes from 
that fact that people can make probability judgments rapidly and easily, and 
typically do not have access to the reasons behind their estimations (using our 
experiment as an example, if we asked a participant why they judged that rain 
had 70% probability of occurring, they would not be able to give any answer 
beyond saying that that was their estimate based on their experience). Further 
support comes from experimental results showing that animals effectively judge 
probabilities (for instance, the probability of obtaining food from a given source) 
and that their judged probabilities are typically close to optimal (for example, 
see 



Kheifets and Gallistel 2012). 



We also expect this automatic probability estimation process to be subject 
to occasional intervention by other cognitive systems, and particularly by con- 
scious, symbolic and slow 'System 2' processes that can check the logical validity 
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of estimates produced by the probability module. We expect this type of inter- 
vention to be both relatively rare, and effortful. To quote one participant in an 
earlier experiment where participants had to choose between betting on a single 
event or on a conjunction containing that event: 'I know that the right answer 
is always to bet on the single one, but sometimes I'm really drawn to the double 
one, and it's hard to resist'. 

4.1 Further predictions 

It should be clear that our prediction for the derived expression Xe(A, B) holds 
because the associated expression X(A,B) sums to 0, and because there are 
an equal number of positive and negative terms in the expression (these two 
requirements are necessary to cancel out the d or noise terms in the expression). 
In this section we give two other expressions where these requirements both 
hold, and so for which the same prediction follows. 

Consider an experiment where we ask people to estimate, for any triplet of 
events A, B and C, the probabilities of those events and all their conjunctions 
and disjunctions. As before we can compute a derived sum 

X E (A, B, C) = P E (A) + P E (B) + P E (C)-P E (A A B) — P E (A AC)- P E (B A C) 

+ P E (A ABAC)- P E (A VBVC) 

Applying Equation [2] as before we get 

X~e~{A, B, C) =P{A) + d- 2dP(A) 

+ P(B) +d- 2dP(B) 
+ P(C) +d- 2dP{C) 

- P{A A B) - d + 2dP{A A B) 

- P(A A C) - d + 2dP(A A C) 

- P{B A C) - d + 2dP(B A C) 

+ P E {A A B A C) + d - 2dP E {A ABAC) 

- P E {A V B V C) - d + 2dP E {A V B V C) 

or rearranging 

Xe(A, B) = (l-2d){P{A)+P{B)+P(C)-P(AAB)-P(AAC)-P{BAC)+P(AABAC)-P(AvBVC)) 
However, as before probability theory's addition law gives 

P(A)+P(B)+P(C)-P(AaB)-P(AaC)-P(BAC)+P{AaBAC)-P(A\/B\/C) = 
for all events A, B and C, and again we see that 

Xe~(A,B,C) =0 

will hold for all triples of events A, B, C, and so we predict that across a range 
of participants and event triples the value of X E {A, B,C) will be normally 
distributed around a mean of 0. 
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Next consider an experiment where we ask people to estimate, for any pair 
of events A and B, the probabilities of A, B, A^B ('A and not £?') and B—*A 
('B and not A') . In this case our derived sum is 

Y E (A, B) = P E (A) + P E {B) - Pe(A^B) - P E {B^A) 
whose expected value will be 

Ye{A, B) = Pe~(A) + Pe~{A^B) - Pe~{B) - Pe~{B^A) 

From Equation [2] we get 

Ye(A, B) =P{A) +d- 2dP{A) 

+ P(A^B) +d - 2dP(A^B) 
-P(B)-d + 2dP(B) 
-P(B^A) -d + 2P(B^A) 

or rearranging 

Ye(A, B) = (1 - 2d){P{A) + P(A^B) - P(B) - P(B^A)) 

However, from probability theory we have 

P(A) + P(A^B) = P(B) + P(B^A) 

(because each side of the expression is equal to P(A V B)) and so we have 

P{A) + P(A^B) - P(B) - P(B^A) = 

for all events A and B, and again we predict that across a range of participants 
and event triples the value of Y E (A,B) will be normally distributed around a 
mean of 0. 

We could, of course, run experiments testing these two predictions ourselves. 
Given the strong confirmation of our original prediction, however, we are con- 
vinced that these predictions will be confirmed: for us, running those experi- 
ments would in some ways be a waste of time. We feel it would be more useful 
for others to test these predictions. We hope that other researchers take up 
this challenge and gather further evidence that people's mechanisms for judg- 
ing probability are not systematically flawed, but instead are based the rules of 
probability theory (but subject to the influence of noise). 
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